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ABSTRACT 



The objective of this research is to develop a Back-propagation Neural Network 
(BNN) to control certain classes of unknown nonlinear systems and explore the network’s 
capabilities. The structure of the Direct Model Reference Adaptive Controller (DMRAC) 
for Linear Time Invariant (LTI) systems with unknown parameters is first analyzed. This 
structure is then extended using a BNN for adaptive control of unknown nonlinear 
systems. The specific structure of the BNN DMRAC is developed for the control of four 
general classes of nonlinear systems modelled in discrete time. Experiments are 
conducted by placing a representative system from each class under the BNN’s control. 
The conditions under which the BNN DMRAC can successfully control these systems are 
investigated. The design and training of the BNN are also studied. 

The results of the experiments show that the BNN DMRAC works for the 
representative systems considered, while the conventional least-squares estimator 
DMRAC fails. Based on analysis and experimental findings, some general conditions 
required to ensure that this technique works are postulated and discussed. General 
guidelines used to achieve the stability of the BNN learning process and good learning 
convergence are also discussed. 

To establish this as a general and significant control technique, further research is 
required to obtain analytically, the conditions for stability of the controlled system, and 
to develop more specific rules and guidelines in the BNN design and training. 
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I. INTRODUCTION 



A. OBJECTIVE 

The objective of this thesis research is to develop a Back-propagation Neural 
Network (BNN) to control certain classes of unknown, nonlinear dynamical systems and 
to explore the network’s capabilities. Discrete-time models, which readily describe many 
real world systems, are used to represent the unknown nonlinear systems for the purpose 
of analysis and simulation. 



B. NEURAL NETWORKS IN ADAPTIVE CONTROL 

Linear control theory is a very mature field. Since the beginning of this century, 
both necessary and sufficient conditions for the stability of Linear-Time-Invariant (LTI) 
systems have been established and rigorously proven. As the result, many powerful and 
well-established techniques (e.g. state-feedback) have been developed to design 
controllers for LTI systems which will achieve any desired system response or any 
specified robustness. In contrast, the conditions for stability of most nonlinear and time- 
varying systems can only be established, if at all possible, on a system-by-system basis. 
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Hence, general control design techniques, even just to achieve stability, are still not 
available for many classes of nonlinear systems. 

From the fifties up to the late seventies, major advances were made in the 
identification and adaptive control of LTI systems with unknown or time varying 
dynamics [Ref. 1]. Many adaptive control techniques, for which global stability 
is assured, have been developed by assuming the system to be LTI, and applying well- 
established results from linear systems theory and parameter estimation. However, the 
original targets of these techniques were actually systems with slowly varying 
parameters. These systems belong to a significant class of nonlinear systems. The 
controllers are also nonlinear systems by themselves. Nonetheless, limited advances have 
been made to address the adaptive control of more general classes of nonlinear systems. 

Recently, the use of neural networks for parametric identification and adaptive 
control of certain general classes of nonlinear systems, based on the indirect adaptive 
control structure, has been suggested [Ref. 2]. In that approach, a neural network 
is first trained to emulate an unknown, nonlinear Single-Input-Single-Output (SISO) 
system. Then the errors between the system output and a desired reference model output 
are back-propagated through the trained neural network emulator to obtain the 
contributing control input error. Based on a suitable minimization function of the control 
input error, a neural network controller is trained to control the system so that it behaves 
like the desired reference model. Simulation results have shown that neural network- 
based indirect adaptive control of large classes of nonlinear systems is not only feasible, 
but seems quite promising as a general technique. 
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The stated objective of this thesis research is to further explore, analyze and 
develop the neural network-based adaptive controller. Specifically, the use of neural 
networks as a direct adaptive controller for some general classes of nonlinear systems 
shall be considered. Unlike the indirect adaptive control approach, only one neural 
network, instead of two, shall be used to learn the unknown control structure and 
parameters directly. The same neural network estimator shall then be used as the 
controller. The development of a neural network estimator-controller is the key issue 
addressed in this thesis. 



C. BASIC CONCEPTS OF ADAPTIVE CONTROL 

Adaptive control has been applied to many areas such as robot manipulation, ship 
steering, aircraft control, chemical process control and bio-medical engineering. The 
applications are mainly aimed at handling parameter variations (slowly time-varying) and 
parameter uncertainties in the system under control. 

In adaptive control, the basic idea is to combine an on-line parameter identification 
process with control system design calculation based on the estimated parameters and the 
required control law to implement the controller. The general structure of an adaptive 
control system is shown in Figure 1. 

Consider the adaptive control of an unknown linear time invariant (LTI) system. 
One scheme is to parameterize the system, for example, by a linear state-space model 
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{A,B,C,D} or a transfer function H(-) with unknown parameters. These parameters are 
then estimated on-line by a suitable estimator. Based on the estimated parameters, 
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Figure 1. General Adaptive Control Structure. 



appropriate design calculations can be performed on-line to implement the chosen control 
law. This class of algorithms is commonly referred to as indirect adaptive control . Figure 
2 shows the structure of an indirect adaptive control system. 

Alternatively, it may be possible to parameterize the unknown system directly in 
terms of the required control parameters (e.g. the state-feedback gains) to implement the 
chosen control law. In this case, the on-line estimator would generate the estimates of 
the unknown control parameters, and then uses them directly for the control. The need 
for design calculation on-line is therefore eliminated. This class of algorithms is called 



4 



direct adaptive control . The structure of a direct adaptive control system is shown in 
Figure 3. 



71 




DESIGN OBJECTIVE ESTD. PARAMETERS 



Figure 2. Indirect Adaptive Control Algorithm. 



DESIGN OBJECTIVE 




Figure 3. Direct Adaptive Control Algorithm. 



Many different methods have also been used to specify the desired behavior or 
performance of a system under adaptive control. One very common scheme is the model 
reference adaptive control. The basic idea is to design the adaptive control system (be 
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it direct or indirect) so that the closed loop system behaves like the specified reference 
model. 

We see that a key component of an adaptive controller is the parameter estimator. 
Many parameter estimation schemes have been devised and employed in adaptive control. 
However, it is important to note that most existing techniques generally require a linear 
parameterization of the system, i.e., parametric uncertainties must be expressed linearly 
in terms of a set of unknown parameters. Such parameterization of the system is usually 
in a form of a regression equation which is linear in the parameters. In linear systems, 
the regressor can usually be formed using only linear functions of the state measurements 
or observations from the systems, with the unknown parameters as coefficients. 
However, in nonlinear systems, nonlinear functions of the measurements or observations 
are generally required. Hence, to use current estimation techniques requires that these 
nonlinear functions are known. However, with unknown nonlinear systems, this will not 
be the case. Hence, the use of neural network as a generalized estimator is proposed in 
such a situation. 

In order to develop a neural network-based direct model reference adaptive 
controller (DMRAC) for certain classes of unknown, nonlinear systems, the design of 
a DMRAC for unknown LTI systems shall be first reviewed and analyzed in detail. 
Based on the same control structure, the neural network shall be employed to extend the 
control to nonlinear systems. 
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D. ANALYSIS OF A DMRAC FOR UNKNOWN LTI SYSTEMS 



Consider an LTI system described by an ARM A model, 

A(q)y(t) = B(q)u(t) , (M) 

with A(q) and B(q) being polynomial operators' with unknown coefficients. A(q) is 
assumed, without loss of generality, to be monic and degree[/4(I(7J] = n > degree[5(i7J] 
= m. For the direct MRAC design, the following assumptions are required: 

1. The upper bound on the system order (i.e. maximum degree of A(q), n) is known. 

2. The system has no hidden unstable modes and has a stable inverse. 

3. The relative degree of A(q ) and B(q) (i.e. n - m) is known. 

The design objective is for the closed loop system to track a reference model 

D(q)y(t) = v(0 , (1-2) 

where D(q) is the monic characteristic polynomial operator of the desired system, and 
v(t), an external input. Let the degree of D(q) be r. It is well known that with linear 
state-feedback, all n poles of the closed loop system can be placed anywhere in the 
complex plane (provided the system is controllable). Hence to achieve model tracking by 
state-feedback, r out of the n poles of the closed loop system must be placed to match 
those of the reference model. The m unwanted zeros of the open loop system must also 



1 The argument q of the polynomials can be interpreted as the forward time-shift operator in 
discrete-time modelling or as the Laplace s-operator in continuous-time modelling. 
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be canceled by the remaining poles of the closed loop system. Hence r must be equal to 
(n - m). For proper pole-zero cancellation, a stable inverse system is also required. 

Very often, only u(t) and y(t) are accessible while the other states required for full- 
state feedback are not. Hence, in these cases, an observer is required. By employing a 
Luenberger observer or a steady-state Kalman filter, it can be shown that the combined 
observer-state-feedback system yields the following structure for the feedback controller, 

u(t) = ^-uit) + -^ly(f) + v(r) , (1-3) 

a(q) a(q) 

where a (q) is the monic characteristic polynomial operator of the observer. It can be 
chosen arbitrarily, provided it is a stable system of degree n. Hence, degree[A(q)] = n 
must be known. The polynomial operators h(q) and k(q) are the feedback polynomial 
operators and have parameters which are determined by the unknown system, the 
observer and the reference model characteristics. It can be shown that degr ze.[h(q)] < 
(n - 1) and degr o&[k(q)] < (n - 1). 

1. Controller Parameters 

To obtain the unknown control parameters in terms the parameters of the system, 
the observer and the reference model, we introduce first the notion of the partial state 
z(t) [Ref.3], in which we represent the system of equation (1-1) as 



8 



Mq)z(t) = u(t) 



(1-4) 



y(t) = B(q)z(t) . 

Combining equations (1-3) and (1-4), we can easily express the closed loop dynamic as 

[a(q)A(q) -h(q)A(q) -k(q)B(q)]z(t) = a(^)v(r) , 

y(t) = B(q)z(t) . 



To obtain the desired closed loop behavior, the equality 

a(q)A(q) - h(q)A(q) - k(q)B(q) = -}-a(q)D(q)B(q) (1-6) 

b x 

must be satisfied so that the closed loop system has r of its poles coincide with those of 
the reference model. The remaining poles must cancel the open loop zeros, so that the 
closed loop dynamic is the same as the reference model’s, apart from the scaling factor 
bj on the reference input v(t). 

Re-arranging equation (1-6), the following Diophantine equation is obtained 
[Ref. l:pp 508-510] 



h(q)A(q) + k(q)B(q) = a(q) 



A(q) - D(q)B(q ) 



(1-7) 



The left side of equation (1-7) is of degree < (2n - 1), while a(q)A(q) is of degree 2 n. 
Hence the factor 1/6; is needed to ensure that (l/b!)a(q)D(q)B(q) is monic and thus 
eliminating the q 2 " term on the right side of the equation. If A(q) and B(q) are relatively 
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co-prime (i.e. there is no pole-zero cancellation), then a unique solution for h(q) and k(q) 
is guaranteed to exist 2 . 

2. Parameter Estimation 

Since A(q) and B(q) are unknown polynomial operators, an estimator is required 
to estimate the system parameters in order to implement the controller using the 
estimated parameters. In the following development, on-line estimation based on a 
particular regression form shall be used to recursively estimate the controller parameters 
directly. 

Applying the polynomial operator in equation (1-7) to the partial state z(t), the 
following regression equation, 

a(q)u(t) = h(q)u(t) + k(q)y(t) + -}-a(q)D(q)y(t) (1-8) 

b i 

is obtained. Then using q as the forward time shift operator and the filtered input and 
output signals defined by 

q- n ct(q)y F (t) = y(t) 9 

q~ n a(q)u F (t) = u(t) , 

a more convenient form of the regression equation (1-8) is obtained in equation (1-10). 



2 The left hand side of equation (1-7) can be cast into a Sylvester matrix multiplied by the 
parameter vector consisting of the unknown coefficients of h(q) and k(q). The Sylvester matrix is 
non-singular if A(q) and B(q) are relatively co-prime [Ref.3:p. 159] and a solution for the parameter 
vector is guaranteed in this case. 
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or 



q' r u(t ) = q-^ r >h(q)u F (t) + q-^ r) kiq)y F (f) + , (1-10) 

b. 



q- r u(t) = m T 0 O , 



where 



m = 



' u F (t-r- 1) ' 
u F (t-r-2) 




K ' 


u F (t-r-n + 1) 
y f (r-r-l) 
y F it-r- 2) 


0 = 


Vd 

k x 

K 


y F (t-r-n+ 1) 
Q ~ r D(q)y(t) . 




hn~ 1) 
1 

b t . 



(i-ii) 



Equation (1-11) is a realizable linear in the parameter regression equation with a linear 
regressor <b(t). In this form, many standard recursive estimation techniques can be used 
to estimate the unknown parameter vector 0 O . The following estimate Q(t) of 0 O is 
obtained by applying the recursive least-squares estimation technique 3 as follows: 

0(r+l) = © ( r) ♦ , (1-12) 

1 + $ T (t)P(f)$(f) 



3 The value of P(0) to start the recursion is discussed in most texts on recursive least squares 
estimation. 
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P(r+1) = P(t) - 

1 + <£> T (t)P(t)<£>(t) 



(M3) 



Now the control equation (1-3) can be rewritten as 



u(t) = q-"h(q)u F (t) + q- n Kq)y F (t) + -i-v(0 . 

b i 



This can be further rearranged as 



(1-14) 



u(t) = $ c (t) T 0 O , 
where 

' u F it- 1) 

«V2) 



® e (0 = 



u F (t-n + 1) 
y F (r-l) 
y F (t- 2) 

y F (r-n+l) 

v(0 J 



(1-15) 



which is identical in structure to the regression equation (1-11). Notice that equation (1- 
15) has the same parameter vector 0 O as equation (1-11). $ c (t) is also identical to $(t) 
except for the time shift (f and the term v(t) replacing D(q)y(t). Therefore, with identical 
structure as the estimator, the controller can be directly implemented without the need 
for an intermediate control design calculation. In the control phase, the current estimate 
0(7) of 0„ is used to generate u(t). 
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3. Summary 

All the necessary steps from performance specification to the design of the 
DMRAC for unknown LTI systems have been developed. In summary, the design and 
implementation procedures are: 



1. The observer characteristic polynomial a(q) of degree n and the desired reference 
model 1/D(q) of degree (n - m) are first chosen. 

2. The closed loop system output y(t) is filtered by the inverse reference model to obtain 

D(q)y(t). (t) and if (t) are obtained by filtering the input and output signals, u(t) and 

y(t), respectively, by the observer (l/fq' n a(q)]). 

3. The vector $(t) is formed as shown in equation (1-11) and used as input to the 
parameter estimator. On-line estimation of the parameter vector 0 O can be performed 
using equations (1-12) and (1-13). 

4. The control signal u(t) for the closed loop system is generated using equation (1-14) 
and the estimated parameter vector Q(t) (instead of OJ. 



Figure 4 illustrates the estimation and control algorithm of the DMRAC. Note that 
the block Q(t) is a linear associative memory with recursive estimation updates to 
minimize mean square errors between u(t) and u(t) = <b(t) T Q(t). 

Appendix A contains a worked example of the design of a DMRAC for an 
unknown LTI systems. Software simulations are conducted to show how the DMRAC 
can be implemented and how it works. MATLAB 4 software environment is employed 
in all the software simulations conducted. 



4 MATLAB® is a registered trademark of The MathWorks, Inc. 
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®c(‘) 




u(t) 




U (t) = 4> c (t) T e(t) 
u(t-r) = O(t) T 0o 




Figure 4: Estimation and Control Algorithm of the DMRAC 



E. ADAPTIVE CONTROL OF NONLINEAR SYSTEMS 

Many variations of the adaptive control technique analyzed above have been also 
developed to handle different assumptions about the unknown LTI systems 
[Ref.4], Since most of these techniques deal with linear systems, simple linear 

functions of the measurements or observations, such as)f(t), y F (t-l if (t), if (t-1) 

(assuming a SISO system) are always sufficient to form the regressor vector $(t) to give 
a regression equation which is linear in the parameters. 

However, with nonlinear systems, the use of nonlinear functions of the 
measurements or observations in the regressor vector becomes almost always necessary 
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in order to keep the regression equation linear 5 . Therefore it is necessary to know the 
exact nature of these nonlinear transformations in order to form the linear regressor to 
allow the use of standard parameter estimation techniques. Chapter 5 of [Ref.5] 
provides more details on the use of standard parameter estimation techniques for 
nonlinear systems. 

Since the nonlinear system to be controlled is assumed unknown, the appropriate 
nonlinear regressor required by the estimator is unknown. Therefore, the conventional 
approach in using standard parameter estimation technique such as least squares 
estimation cannot be used. It has been shown in [Ref. 6] that a neural network can 
learn to emulate any continuous function. The idea then is to replace the linear 
associative memory of 6(t) with a neural network. The neural network shall be taught 
to emulate the appropriate nonlinear controller in the same manner as the recursive least 
squares estimator is used in the DMRAC for LTI systems. The specific structure of the 
neural network-based direct model reference adaptive controller for certain classes of 
nonlinear systems is developed in the next chapter. The performance of the neural 
network as a DMRAC is investigated experimentally in Chapter III. 



5 This implicit requirement arises from the fact that existing estimation techniques generally 
require a linear parameterization of the system. 
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II. BNN DIRECT MODEL REFERENCE ADAPTIVE CONTROL 



A. APPLICATIONS OF NEURAL NETWORKS 

A large variety of artificial neural networks has been developed and employed in 
numerous applications [Ref. 7]. Successful applications of artificial neural networks 
have been developed in such areas as pattern recognition, speech and natural language 
processing, image compression, functional optimization, and even financial and economic 
system modelling. Artificial neural networks have also been highly touted for control 
engineering applications with early experiments such as the self-learning broomstick 
balancer [Ref. 8] and the recent neural network truck backer-upper [Ref. 9]. 

A neural network usually consists of a large number of simple processing elements, 
known as neurons. Each neuron has a number of inputs, each associated with a synaptic 
weight as shown in Figure 5. It usually performs only very simple mathematical 
operations: 

• each input (including a fixed bias) to the neuron is multiplied by the associated 
synaptic weight. 

• the results of the multiplications for all the inputs are summed. 

• the summand is then mapped to the output of the neuron through a nonlinear 
function Tf-J. Typically, T[J is a monotonically increasing function (e.g. tanh[-]). 

The first two operations is actually a scalar dot-product between the inputs and 
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the associated synaptic weight vector of the neuron. The neurons are often interconnected 



in layers, in a predefined manner. 

The most distinctive and appealing 
feature of many neural networks is that 
they learn by examples. Learning in the 
context of artificial neural network, is 
achieved through adapting the synaptic 




Figure 5: A Neuron. 



weights of the neurons. The synaptic 

weights then serve as a form of associative memory mapping the inputs of the neural 
network to its outputs. Based only on pre-assigned learning rules, the neural network can 
hence derives its functionality through learning by examples rather than through the 
traditional programming approach employed in traditional von Neumann machines. 
Hence, neural networks provide an approach that is closer to human perception and 
recognition than most other information processing approaches in use today. 

Currently, the most popular and commonly used neural networks for control system 
design is the Back-propagation Neural Network (BNN). Its popularity stems from the fact 
that the BNN implements a learning procedure, known commonly as the generalized delta 
rule [Ref. 10], that allows it to learn to emulate a very large class of nonlinear 
functions. 

The structure of the BNN will be discussed in detail in the next section. This is 
followed by a description of the four general classes of SISO nonlinear systems 
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considered for control by a BNN DMRAC. Finally, the structure of a BNN DMRAC for 
each class of these nonlinear systems is established. 



B. ANALYSIS OF BACK-PROPAGATION NEURAL NETWORK 

A back-propagation neural network is a multi-layer, feed-forward network which 
has an input layer, an output layer and at least one hidden layer. Neurons are found in 
the output and hidden layer(s) while the input layer has only input connections feeding 
the neurons in the first hidden layer. Figure 6 shows a multi-layer back-propagation 
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Figure 6. A Back-propagation Neural Network 
network with two hidden layers. In the back-propagation neural network, the signal flows 
from input to output layers. There is no feedback or even interconnection between 
neurons in the same layer. There is usually also a bias input for each neuron with an 
associated non-zero synaptic weight. 
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To describe mathematically the learning process the BNN uses, we first define w jk l ‘ J 
as the connection weight for the path from the j* neuron in (i-l)* layer to the k* neuron 
in i 01 layer. Also define*, as the j* input to the neural network. Then the BNN in Figure 
6 can be represented mathematically as 

y m = , ( 2 - 1 ) 

where x = {*,} is the vector of all the inputs to the BNN. W i} — {wj' 1 } the synaptic 
weight matrix of the i* layer formed from columns of synaptic weights associated with 
the inputs of each neuron, y li] = {y/ 7 } is the vector of all outputs in the i* layer. Next 
we define also zf 1 as the summation of weighted inputs of the j 01 neuron. In the learning 
process, the BNN adjusts the synaptic weights W iJ for all i, to minimize a suitable 
function of the error between the output y lN] and a desired output yd = { yd k } for a N- 
layer BNN. The most common error function used is 

E = ^ E (y d k~yf * 

z Alik 

where k is the index spanning all the output neurons. This minimization is performed for 
each set of input vector given to the BNN. Other forms of error functions, including the 
sum of the absolute errors, can also be used. 

The BNN implements a modification of the gradient descent algorithm (also known 
as the least-mean-squares method, LMS) to update each synaptic weight at time t + 1 
with 
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(2-3) 
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where n and v are scalars representing the learning rate and the momentum rate. The 
learning rate is equivalent to the step-size parameter in the conventional LMS algorithm. 
Like the LMS algorithm, too large a learning rate often leads to instability of the learning 
system while too small a value would result in a very slow learning process. The use of 
a momentum term has been found to speed up the learning process considerably. It 
allows a larger learning rate yet avoids the point of instability. For the i* layer, 
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where the local error vector ej' 1 is given by 



(2-4) 



[*1 w/ [*]\ [* + l] [* + l] c'x 

e ) = r (^ rE e k * w jk • (2-5) 

All k 

Equation (2-4) is a direct application of the chain rule in differential calculus. At the 
output layer (say, N* layer), 



^=4D*K-^r) ■ < 2 - 6 > 

Once = {ef* 1 } at the output layer is obtained, then e IN1] , e lN ' 21 ,... can be recursively 
computed using equations (2-4) and (2-5). The weights can be updated using equation (2- 
3). Note that different learning rates and momentum rates can be used in different layers. 
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The equations (2-3) through (2-5) describe mathematically the error back-propagation 
mechanism from which the BNN derived its name. Figure 7 describes the learning 
process diagrammatically. T’(-) is the first derivative function of T(-) and ir represent the 
term-by-term products of the two sets of inputs. 

As proposed earlier, a direct model reference adaptive controller shall be built by 
replacing the linear associative memory block of the DMRAC (see Figure 4) with a 
BNN. As an initial proof of the concept, an experiment was conducted by replacing the 
least-squares estimator with a BNN directly in the DMRAC for unknown LTI systems. 
The results of this experiment are shown in Appendix B together with the programs 
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Figure 7. Back-propagation Neural Network Learning 
developed for the simulations. The results indicate that the BNN in the DMRAC structure 
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